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[Document] SPECIFICATION 
[Title of the Invention] ELECTRONIC CAMERA 
[What is Claimed is:] 
[Claim 1] 

An electronic camera characterized by con^rising an Imge 
sensing element composed of a single panel and having a pixel 
corresponding to at least three different color information, 
characterized in that 

the camera further comprises interpolation means for 
generating a plane that corresponds to number of all pixels by 
interpolation processing based on an approximate expression 
including a polynomial of at least ard-order with use of pixel 
values having color information same as that of the pixel, 

[Claim 2] 

The electronic camera according to claim 1, characterized 
in that the interpolation means further comprises a table 
storing an interpolation coefficient, and the interpolation 
means interpolates the pixel using the interpolation 
coefficient stored in the table. 
[Detailed Description of the Invention] 
[0001] 

[Technical Field of the Invention] 

The present invention relates to an electronic camera 
which can electronically sense an object image and display and 
record obtained image information, in particular, relates to 
an electronic camera capable of processing a pseudo 3-CCD 
processed image with use of an image sensing element composed 
of a single panel. 
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[0002] 
[Prior Art] 

An image pickup apparatus is disclosed, in this 
apparatus, to avoid a false signal at an edge portion in pixel 
interpolation, two images are generated by spatial pixel 
shift, and a synthesized image thereof is obtained. (Jpn, Pat- 
Appln, KOKAI Publication No. 10-248069) 

[0003] 

[Object of the Invention] 

The present invention has its object to provide an 
inexpensive and high-quality- image electronic camera by pseudo 
3-CCD processing an image photographed by an image sensing 
element composed of at least a single panel. 

[0004] 

[Means for Achieving the Object] 

The present invention takes a following measure to solve 
the above objects- 
[0005] 

An electronic camera according to the present invention 
is characterized by comprising an image sensing element 
composed of a single panel and having a pixel corresponding to 
at least three different color information, in which the 
electronic camera further comprises an interpolation means for 
generating a plane that corresponds to number of all pixels by 
interpolation processing based on an approximate expression 
including a polynomial of at least 3rd-order with use of pixel 
values having color information same as that of the pixel. In 
this manner, pseudo 3-CCD processing can be performed by high- 
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quality interpolation/ an inexpensive and high-quality image 
electronic camera can be provided 
[0006] 

In the above-described electronic camera, the 
interpolation means further comprises a table storing 
an interpolation coefficient, and the interpolation means 
interpolates the pixel using the interpolation coefficient 
stored in the table. Interpolation calculation is executed by 
looking up the table prepared in advance to avoid complex 
calculation. Hence, this allows high-speed processing and 
requires no high-performance arithmetic element. 

[00071 

[Embodiment of the Invention] • 

An embodiment of the present invention will be described 
below with reference to the accompanying drawing. 
[0008] 

FIG. 1 is a schematic block diagram showing the system 
arrangement of an electronic camera according to an embodiment 
of the present invention- The schematic arrangement of the 
electronic camera according to the present invention will be 
described with reference to FIG. 1. 

[0009] 

An object image that has passed through an image sensing 
lens system 11 is converted into an electrical signal by an 
image sensing element 12. The electrical signal converted by 
the image sensing element 12 is converted into an analog image 
signal by an image sensing circuit 13 and then into a digital 
image signal by an A/D converter 14. This digital image 
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signal is subjected to a series of image processing operations 
and then recorded in a detachable memory 20 as an external 
memory (e.g./ a flash memory or smart medium) through an 
interface (I/F) 21* The detachable memory 20 is noxmally 
inserted into a card slot 22. The electronic camera also has 
an internal memory 30 (e.g., a RAM (Random Access Memory)) 
which operates at a high speed and is used as a working memory 
in image compression and expansion, or an image storing means, 
that is, a high-speed buffer for temporarily storing various 
image data. In the present invention, the internal memory 30 
has a memory area 31 for interpolation processing. This 
memory area 31 can be prepared independently of the internal 
memory 30 or incorporated in an interpolation calculation 
circuit (or IC) . 
[0010] 

A compression/expansion section 40 compresses the digital 
image signal or expands (elongates) the compressed image 
signal, 

[0011] 

The electronic camera normally has an LCD 50 (liquid 
crystal display device) for displaying an image. The LCD 50 
is used to confirm an image recorded in the detachable memory 
20 or display an image to be sensed. An image can be 
displayed on the LCD 50 by temporarily loading image 
information from the internal memory 30 to a video memory 51 
and converting the image information into a video image by a 
video output circuit 52. The output from the video output 
circuit 52 can be output. to an external display device as a 
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video image through an external tenainal 53 for video output. 
[0012] 

A system controller 70 systematically controls 
the sections of the electronic camera. The detailed functions 
of the system controller 70'wUl be described later. The 
system controller 7 0 receives an input from an operation 
section 73 formed from a release button and senses an image in 
accordance with the operation of the release button, or 
requests an image processing circuit (not shown) to process an 
image. If the light amount in sensing an object is too small, 
the system controller 70 requests and controls an electronic 
flash light-emitting section 71 to turn on the electronic 
flash for image sensing. The system controller 70 also has an 
iiaage sensing distance detection section (not shown) having a 
function of detecting the distance from the object. The 
operation section 73 can set various modes. The mode setting 
is displayed on a mode LCD 72. 
[0013] 

An external interface (external I/F) 61 is connected to 
an external input/output terminal 60 to input/output data 
from/ to an external device. The external input/output 
terminal 60 is connected to, e.g., a personal computer to 
transfer an image in the detachable memory 20 to the personal 
computer or receive image data from the personal computer. 
[00141 

The sections of the electronic camera are basically 
driven by a battery. Power supplied from a camera battery 61 
through a power supply section SO drives the sections of the 



PAGE 22/39 * RCVD AT IZim 11:26:08 AM [Ea^em Mard rimep SVR:^^^^^ 



Dec. 2.2004- !1:28AM FRISHAUF & PARTNERS 



No. 1446 P. 23/39 



- 6 - 

camera. The camera battery 81 can be charged under the 
control of the power supply section 80. 
[OOlS] 

FIGS. 2(a) and 2 (b) are graphs showing one-dimensional 
interpolation models. Conventionally, linear interpolation of 
calculating an output value at a desired position using a line 
connecting two points is generally used, as shown in 
FIG. 2 (a). In this method, although only two points suffice 
as positions having known output values necessary for 
calculation, the output value obtained is merely the 
proportional average between the two points. For example, 
even when the maximum value or minimum value is present 
between the two points, it cannot be detected. In the present 
invention, to improve the interpolation accuracy, an output 
value at a desired position is obtained by an approximate 
expression using a multi-order polynomial of at least 
3rd-order. FIG. 2 shows an example in which the coefficients 
of a 3rd-order polynomial are obtained from values at four 
points, and an output is obtained by substituting position 
data to an approximate expression using the obtained 3rd-order 
polynomial . This interpolation by an approximate expression 
using a 3rd-order polynomial is also called "cubic 
interpolation". Referring to FIG. 2, the coefficients of the 
3rd-order polynomial are obtained from the output values at 
four positions (n-1), n, tn+l) and (n+2}, the output value at 
a position x' is obtained from the 3rd-order polynomial, 
thereby obtaining an interpolation value at the desired 
position. If this processing is done using linear 
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interpolation, the position (n+1) is obtained as the position 
having the maximum value. Hence, no accurate position can be 
obtained, unlike the present invention. 
[0016] 

FIG- 3 is a block diagram showing the arrangement of the 
interpolation calculation circuit 90 in the present invention. 
The interpolation calculation circuit 90 comprises 
an interpolation position calculation section 91, 
interpolation position correction section 92, interpolation 
coefficient table 93, interpolation calculation section 94, 
and buffer memory 31 . The functions of the interpolation 
position calculation section 91 to the interpolation 
calculation section 94 are controlled by the system controller 
70. More specifically, the interpolation calculation circuit 
90 performs the following operation, 

[0017] 

Original image data from the internal memory 30 is input 
to the interpolation position calculation section 91 and 
interpolation calculation section 94. On the basis of the 
input data, the interpolation position calculation section 91 
calculates, e.g., the interpolation position between the 
point n and the point (n+l) 'shown in FIG. 2(b) • Next, on the 
basis of data from the memory area 31, the interpolation 
position correction section 92 corrects the interpolation 
position to a point closest to the point x\ which is one of 
points that divide the section between the point n and the 
point (n+l) into, e.g., 16 equal parts. By correcting the 
interpolation position, the output at the corrected 
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interpolation position can t)e calculated using the 
interpolation coefficient table 93 prepared in advance. 
Hence/ the output value at the interpolation position can be 
calculated fast without any complex calculation. A3 for an 
error by correction of the interpolation position, the 
interpolation coefficients are given in the table, and 
therefore/ it is impractical to excessively finely set 
interpolation positions because the data quantity of the table 
increases- Relatively finely, e,g., equally dividing a 
section into 16 portions suffices in terms of accuracy and is 
practical because the data quantity is not so large. 
[0018] 

FIGS. 4 (a) to 4(c) are views for explaining pseudo 3-CCD 
processing using interpolation calculation by an approxiiuate 
expression using a higher-order polynomial. 

As shown in FIG. 4(a), a single image sensing element has 
a predetermined color layout of R (red) , G (green) , and B 
(blue) in units of pixels. One pixel corresponds to one 
color. In the present invention, using interpolation by an 
approximate expression using a higher-order polynomial, a red 
interpolation value is embedded in pixels without red color 
information, a green interpolation value is embedded in pixels 
without green color information, and a blue interpolation 
value is embedded in pixels without blue color information, 
thereby implementing pseudo 3-CCD processing. This process is 
shown in FIG. 4 (b) . Thereafter, the pixel planes shown in 
FIG. 4(b) with pixel planes shown in FIG- 4 (c) are replaced on 
the basis of relationships: 
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Y = aR + bG + cB 
Cb - a (B-Y) 
Cr - 0 (R-Y) . 
[00191 

FIGS. 5 (a) to 5(c) are views for explaining the 
interpolation method for the R, G, and B colors. Referring to 
FIGS, 5(a) to 5(c), each of (RJ , (G) t and (B) represents that 
the color information is not present. FIG. 5(a) shows 
interpolation of R (red) . In this case, R (red) color 
information at neighboring points ®, ®, and @ having R (red) 
color information can be interpolated using data (16 R data) 
indicated by a brc]cen line A in fig. 5(a) . R (red) color 
information at points (J), (I), and ® can be obtained on the 
basis of data indicated by a broken line B in FIG, 5(a). 
Pieces of color information at all squares are obtained by 
such interpolation. For G (green) , since pixels having G data 
and pixels having no G data are alternated, for example, (G) 
data of a hatched pixel is obtained using 16 data represented 
by a broken line C. B (blue) shown in FIG. 5(c) has the same 
layout as that of R (red), so data at a no-pixel portion can 
be interpolated by the same procedure as that for R (red) . In 
this case, data for 16 pixels (16 data are used 
for two-dimensional interpolation using a 3rd-order 
polynomial, as described above) are prepared as an 
interpolation coefficient table. Hence, a small table 
suffices as an interpolation coefficient table. In addition, 
since interpolation coefficients are given by the table, and 
no complex calculation is necessary, pseudo 3-CCD processing 
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can be implemented using a small circuit^ 
(0020] 

FIGS. 6 (a) to 6(c) are views for explaining another 
interpolation method for the G, and B colors. In the 
example shown in FIGS. 5(a) to 5(cl, to interpolate the R, G, 
and B colors, for, e.g./ the R (red) region A shown in 
FIG. 5(a), the interpolation value is obtained from 7X7 
pixels (basically, a region containing 8X8 pixels is 
extracted). For G (green) shown in FIG- 5(b), a rhombic 
region is set to obtain the interpolation value. When the 
interpolation value is obtained by the region setting method 
shown in FIGS. 5(a) to 5(c), R (red) and B (blue) can use the 
same effective band, though the effective band of G (green) is 
different from that of the remaining colors. This 
interpolation method is therefore not preferable. To avoid 
this, in the present eiobodiment, all the R, G, and B colors 
are interpolated using 4X4 pixels. FIG. 6(a) shows 
interpolation of R (red), FIG. 6(b) shows interpolation of G 
(green), and FIG. 6(cl shows interpolation of b (blue). For 
example, referring to FIG. 5(a), when 4X4 pixels are 
extracted, the extracted region has at least four R (red) 
pixels (i.e., four data). Using the four data, the 
interpolation value at a position O shown in FIG. 6(a) is 
obtained by an approximate expression using a 3rd-order 
polynomial. G (green) and B (blue) are also interpolated 
according to the same procedure as that for R (red) . Since 
the frequency band changes depending on the position and 
interpolation coefficient, pixel data is corrected by a 
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correction coefficient corresponding to the pixel position, as 
shown in FIGS. 6(a) to 6(g)- Thus in this embodiment, since 
the number of interpolation pixels can be decreased, and the 
R, G, and B color data are generated using the same 
interpolation coefficient on the basis of same 4x4 pixels, 
the same effective band can be used. 
[0021] 

FIG. 7 is a view showing for explaining still another 
interpolation method for the G (green) plane. Referring to 
FIG. 7, a square region is extracted, like R (red) or B 
(blue) , instead of extracting a rhombic region as shown in 
FIG. 5(b). Referring to FIG- 7, for example, data 
interpolated by (G) and that interpolated by [G] are added to 
obtain an interpolation value. This makes it possible to 
generate G (greeni data using the same interpolation 
coefficient in the same pixel region as that of R (red) or B 
(blue) t so the same effective band can be used. In addition, 
since the number of pixels used for interpolation is larger 
than that of the example shown in FIGS. 6(a) to 6(c), the 
accuracy increases. 
[0022] 

FIG. 8 is a schematic view showing the flow of pseudo 
3-CCD processing. The output from the CCD is converted into a 
digital signal by the A/D converter and is branched to two 
signals. One signal is used for edge extraction, and the 
other signal is used for pseudo 3-CCD processing. The signal 
representing the edge of the image is extracted from the 
broadband signal of the image. Pseudo 3-CCD processing is 
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executed using the narrow-band signal of the image. The 
signal representing the extracted edge is synthesized with 
three image signals, and the synthesized signal is siibjected 
to image processing. The process corresponding to *edge 
extraction processing^ performs broadband processing by 
compensating the high bandwidth component to 3-CCD processing 
with an edge extraction processing in this manner. Signal 
used in the edge extraction processing contains many broadband 
con^onents, for example, the G (green) signal generated in the 
embodiment shown in FIG. 5 is used. 
[0023] 

As described above/ according to the present invention, 
for a pixel having, e.g., R (red) pixel information of the R, 
G, and B color information, G (green) and B (blue) pixel 
values are obtained by interpolation by an approximate 
expression using a higher-order polynomial, and for a pixel 
having no R (red) color information, the pixel value is 
obtained by similar interpolation. With this interpolation by 
an approximate expression using a higher-order polynomial, a 
more accurate interpolation value can be obtained than the 
conventional, e.g., linear interpolation, so an inexpensive 
and high-quality electronic camera can be provided. 
[0024] 

The present invention is not limited to the above 
embodiment. In the above embodiment, only an exan^le of a 
one-dimensional 3rd-order polynomial has been described as an 
approximate expression* However, the polynomial can be 
extended to a polynomial of 4th- or higher order. 
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Interpolation by a multi-dimensional polynomial more than a 
two-dimensional polynomial is also possible. If not an 
approximate expression by a polynomial but a more accurate 
approximate expression using another appropriate function 
{e.g., an exponential function) is obtained/ it is very 
effective to use the approximate expression and give the 
interpolation coefficient as a table. 
[0025] 

Various changes and modifications can be made without 
departing from the spirit and scope of the present invention. 
[0026] 

[Advantages of the Invention] 

Following advantages are obtained from the present 
invention. 
[0027] 

According to the present invention, an inexpensive and 
high-quality image electronic camera can be provided by 
enabling pseudo 3-CCD processing, performing high-quality 
interpolation by an approximate expression using a higher- 
order polynomial , 
[0028] 

In addition, interpolation calculation is executed by 
looking up a table prepared in advance to avoid complex 
calculation. Hence, this allows high-speed processing and 
requires no high-performance arithmetic element. 
[Brief Description of the Drawings] 
[FIG. 1] 

A schematic block diagram showing the system arrangement 
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of an electronic camera according to an embodiment of the 
present invention. 
[FIG. 2] 

Graphs showing one-dimensional interpolation models. 
[FIG. 3] 

A block diagram showing the arrangement of an 
interpolation calculation circuit 90 in the present invention. 
[FIG. 4] 

Views for explaining pseudo 3-CCD processing using 
interpolation calculation by an approximate expression using a 
higher-order polynomial . 

[FIG. 5] 

A view showing for explaining the interpolation method 
for each of R, G/ and B colors. 
[FIG. 6] 

A view showing for explaining another interpolation 
method for each of R, G, and B. 
[FIG. 7] 

A view showing for explaining still another interpolation 
method for the G. 
[FIG. 8] 

A schematic view showing the flow of pseudo 3-CCD 
processing . 

[Explanation of Reference Symbols] 
11: Image sensing lens system 
12: Image sensing element 
13: Image sensing circuit 
14: A/D converter 
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20: Detachable memory 

21: Interface (I/F) 

22: Card slot 

30: Internal memcry 

31: Memory area for interpolation processing 

40: Compression/expansion circuit 

50 : LCD 

51 : Video memory 

52: Video output circuit 

53: External terminal 

70: System controller 

73: Operation section 

71: Electronic flash light-emitting section 

61: External interface (external I/T) 

80: Power supply section 

81: Camera battery 
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FIG. 5 A 



1 R 


{R) 


1 ^ 


(R) 


R 


(R) 


R 


(R) 


R ! 


i(R) 


(R) 




hR, 




(R) 


iR) 


(R) 


(R)i 


• R 


[\ 


i R 

w 


(R>^ 




>— 
PR) 


R 


(R) 


R ; 


■(R) 


(R) 


^(R) 


(R) 


(R) 


(R)i 


i ^ 


(R) 




(R) 


R 


(R) 


R 


(R) 


R 1 


i(R) 


(R) 


(R) 


(R) 


(R) 


(R) 


<R) 


(R) 


(R)! 


iR 


(H) 


R 


<R» 


R 


(R) 


R 


(H) 


R 1 


(R) 


(R) 


(R) 


(R) 


(R) 


(R) 


(R) 


(R) 


(R) 



(c) 



(G) 


G 


(G) 


G 


(G) 


G 


(G) 


G 


(G) 


G 


(G) 


G 








G 


(S) 


G 


{G) 


G 


(G) 


''G 


(G) 


G'> 


'j(G) 


G 


(G) 


G 


(G> 


'Q 


(S) 


G 


m 


G- 


<G) 


_G. 


(G) 


:'G 


(G) 


G 


^d)i 


G 


(G) 


g: 


(G) 


G 


(Gy 


.G 


(G) 


G 


(G) 


G,, 


fG) 


G 


(G) 


G 


(G> 


.G 


(G) 


°' 


-'(Q) 


G 


(G) 


G 


(G) 


G 


(Gy> 


vG, 


1G) 


G 


(G) 


G 




<B) 


(B) 


(B) 


(B) 


(B) 


(B) 


(B) 


(B) 


(B) 


(B) 


B 


(B) 


6 


(B) 


B 


(B) 


B 


(B) 


(B) 


(8) 


(B) 


(B) 


(B) 


(B) 


(B) 


(B) 


(B) 


(B) 


B 


(B) 


B 


(B) 


B 


(B) 


6 


(B) 


(B) 


{B} 


(B) 


(B) 


(B) 


(B) 


(B) 


(B) 


(B) 


(B) 


B 


(B) 


B 


(B) 


B 


(B) 


B 


(B) 


(B) 


(6) 


(B) 


(B) 


(B) 


(B) 


(B) 


(B)' 


(B) 


(B) 


B 


(B) 


B 


(B) 


B 


(B) 


B 


(B) 



PAGE 36/39 ' RCVD AT 12l2/20» 11:26:08 AM [Eastern Standard Time] ' SVR:USPT0-EF)(RF*1/2 ' DNIS:872S306' CSID:212 319 5101 * DURATION (inni-ss):08-02 



Dec. 2.2004 - 1! :31AM FRISHAUF h PARTNERS 



No. 1446 P. 37/39 



]6l FIG. 6 













B 




B 




— ( 


>— 






B 




B 



X 



X 







k20 












koz 




) — 

k22 















kio 




k30 


koi 


— c 


k2i 
) — 






ki2 




ki3 


^03 



















kii 




ki3 




— ( 


>— 






k3i 




k33 



[^71 



(a) 



FIG. 7 



(b) 



(c) 





(G) 




(G) 




(G) 




(G) 


CG) 




(G) 




(G) 










(G) 




(G) 




(Q) 




(Q) 


(Q) 




(G] 




(G] 




m 






(G) 




(G) 




(G) 




(G) 


[G) 




to] 




(G) 




[G) 






(G) 




(Q) 




(G) 




(G) 



EDGE EXTRACTION 




PUSEimO 3 -CCD PROCESSING 



IKAGE PROCESSING 



PAGE 37/39 * RCVD AT 1M004 11:26:08 AM [Eastern aandard Time] * SVRiUSPW^^^ 



Dec. 2. 2004 - 11:31AM FRISHAUF & PARTNERS 



No. 1446 P. 38/39 



[Document] ABSTRACT 
{Abstract] 

[Object] To provide an inexpensive and high-quality image 
electronic camera by pseudo 3-CCD processing an image 
photographed by an image sensing element composed of at least 
a single panel. 

[Means for Achieving the Object] An electronic camera that 
comprises an image sensing element 12 composed of a single 
panel and having a pixel corresponding to at least three 
different color information, the camera further comprises an 
interpolation means 90 for generating a plane that corresponds 
to number of all pixels by interpolation processing based on 
an approximate expression including a polynomial of at least 
3rd-order with use of pixel values having color information 
same as that of the pixel, 
[Elected Figure] FIG, 1 
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